IMAGING SYSTEM UTILIZING SPATIAL IMAGE OSCILLATION 
RELATED APPLICATIONS 

[00001] This patent application is a continuation-in-part of International Patent 
5 Application PCT/CA02/00886, entitled "Vision System, Method and Apparatus 
Utilizing Spatial Oscillation", filed June 6, 2002, and claims the benefit of the filing 
date of Canadian Patent Application 2,349,828, entitled "Method, Apparatus, and 
System for Extracting Denoised, High-Resolution Images, Texture, and Velocity 
from a Lower-Resolution, Noisy Detector", filed June 6, 2001. International patent 
10 application PCT/CA02/00886 was published, in the English language, in 
publication document WO 02/099745 A2, on December 12, 2002. 

FIELD OF THE INVENTION 

[00002] The present invention relates generally to imaging systems and 
15 methods, and more particularly to an electronic vision system that obtains high- 
resolution de-noised images and velocities from a lower-resolution detector. 

BACKGROUND OF THE INVENTION 

[00003] The extraction of real-time velocity and noise-free detail from time- 
20 blurred frames of video has been inherently inaccurate. The problem is that 
current image detection technology is based upon raster, frame-at-a-time, or 
arrays of independent pixels for scene capture using light (x-ray, ultraviolet, 
infrared or other medium) integrating pixels. This process is limited in resolution 
by the number of pixels and their dimensions, and cannot avoid integrating noise 
25 into the detection process. Accordingly, attempts have been made to devise an 
improved image detection and processing mechanism. 

[00004] For instance, J.C. Gillete ("Aliasing Reduction in Staring Infrared 
Imagers Utilizing SubPixel Techniques") describes a method of uncontrolled 
micro-scanning for reducing aliased signal energy in a sequence of temporal 
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image frames obtained by periodically sampling an image with a finite array of 
image detectors. Gillete takes a series of discrete low resolution samples of an 
image at a specified undersampling frequency, while spatially oscillating the 
detector between samples, thereby providing a sequence of static image frames, 
5 each having a subpixel offset relative to one another. By comparing the gray- 
scale values of successive image frames, for each image frame an estimate is 
calculated of each subpixel shift that occurs between successive image frames. 
Each image frame in the image sequence is then mapped onto a high resolution 
grid, based on the respective estimated interframe displacement. If the 
10 estimated shift is the same for multiple frames, then the pixel values at the 
overlapping positions are averaged to suppress noise. 

[00005] Since Gillette only calculates an estimate of the subpixel shifts, Gillette 
is unable to determine the portion of the magnitude of the pixel values actually 
attributable to the subpixel shifts. This problem is compounded by the fact that 
15 Gillette averages the values of the pixels in successive frames that have the 
same estimated subpixel shift, thereby precluding removal of those aspects of 
the image frames not attributable to the subpixel shift. As such, the high 
resolution grid would include pixels whose values are not attributable to the 
subpixel shifts (eg. resulted from detector noise). 

20 [00006] Further, Gillette must estimate the subpixel shift in each frame, 
resulting in multiple frame delays for one high-resolution image. Also, the frame 
basis of the method, and the corresponding finite exposure times, results in 
motion blur in each frame as objects traverse the scene. Additionally, given the 
discrete sampling nature of the method, aliasing in time is possible if the 

25 sampling frequency is insufficient for the scene motion. 

[00007] H. Ogmen ("Neural Network Architectures for Motion Perception and 
Elementary Motion Detection in the Fly Visual System") describes a neural 
network model of motion detection in the fly visual system. Ogmen uses center- 
surround opponency as the basis for both directional and non-directional motion 
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detection, both in the center field-of-view and the periphery. However, Ogmen 
only performs statistical neural filtering post-processing of the vision data, 
thereby integrating noise with the vision data, with the ultimate result of reduced 
signal detection. 

5 [00008] For the foregoing reasons, there is a need for an improved electronic 
imaging system. 

SUMMARY OF THE INVENTION 

[00009] The present invention is directed to an imaging system and method 
10 utilizing spatial image oscillation to obtain enhanced image processing of an 
image. 

[00010] In one aspect of the present invention, the imaging system includes: 

an image detector for producing a continuous image signal from an image 
received thereat; 

15 oscillation means coupled to the image detector for inducing a spatial 

oscillation in the image relative to the image detector; and 

a spatio-temporally matching filter in communication with the image 
detector and the oscillation means for providing enhanced image processing of 
the image, the matching filter being configured to filter out aspects of the image 

20 signal not associated with the induced oscillation. 

[00011] In another aspect of the present invention, the imaging system 
includes: 

an image detector for producing a continuous image signal from an image 
received thereat; 

25 oscillation means coupled to the image detector for inducing a spatial 

oscillation in the image relative to the image detector; and 
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a spatio-temporally matching filter in communication with the image 
detector and the oscillation means, the matching filter being configured to filter 
for aspects of the image signal (eg. image edges) that are associated with the 
induced oscillation for providing enhanced image processing of the image. 

5 [00012] In one implementation, the system includes a secondary array of 
opponent center/surround detectors grouped by spatial orientation to provide 
multiple evenly-spaced orientations in the primary array for receiving signal 
inputs at the center/surround detector's center (interpreted as either positive or 
negative depending upon the polarity of opponent surround being passed the 
10 illumination information), and receiving opposing input from primary detectors 
adjacent to a primary central detector, each center/surround detector fed by one 
or more detectors from the primary detector array, therein extracting improved 
contrast, velocity, and relative luminance change information. 

[00013] In an aspect of the present invention, the method involves the steps of: 

15 (i) providing a continuous image signal by spatially oscillating an image 

relative to an image detector, and 

(ii) with reference to the spatial oscillation, filtering out aspects of the 
image signal not associated with the spatial oscillation. 

[00014] In another aspect of the present invention, the method involves the 
20 steps of spatially oscillating an image in one or more dimensions relative to a 
detector array, and filtering for aspects of the signal (eg. image edges) that are 
associated with the induced oscillation. 

[00015] In one implementation, the spatial oscillation step is provided by a 
swept-frequency sinusoid chirp. Further, the method includes the step of 
25 receiving signal inputs at the center/surround detector's center (interpreted as 
either positive or negative depending upon the polarity of opponent surround 
being passed the illumination information), and receiving opposing input from 
primary detectors adjacent to a primary central detector, each center/surround 
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detector fed by one or more detectors from the primary detector array, therein 
extracting improved contrast information to provide real-time spatial motion and 
velocity information. 

[00016] The invention obtains a high-resolution, de-noised image and 
5 component velocities, directly from a lower-resolution detector feeding 
contrast/motion-sensitive opponent center surround detectors, by removing 
spatio-temporal noise, and correcting for pixel-to-pixel differences due to 
differences in detector responses, both in DC offset and gamma variations. As 
well, the invention greatly reduces downstream computational load for intelligent 
10 vision applications, and since this is applied at the detector level, if no active 
transmission, or illumination are involved, it can be purely passive, and therefore 
stealthy. 

[00017] Other aspects and features of the ^present invention will become 
apparent to those ordinarily skilled in the art upon review of the following 
15 description of specific embodiments of the invention in conjunction with the 
accompanying figures. 

BRIEF DESCRIPTION OF THE DRAWINGS 

[00018] These and other features, aspects, and advantages of the present 
20 invention will become better understood with regard to the following description, 
appended claims, and accompanying drawings where: 

[00019] Figure 1 is an overview of a vision system utilizing spatial oscillation in 
accordance with the present invention; 

[00020] Figure 2 is an overview of a vision method utilizing spatial oscillation in 
25 accordance with the present invention; 

[00021] Figure 3 illustrates various proposed illuminated opponent 
center/surround structures with output responses; 
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[00022] Figure 4 illustrates several primary detectors connected to a secondary 
opponent center/surround cell, interleaved with other primary detectors 
connected to another secondary opponent center/surround cell of opposite 
polarity; 

5 [00023] Figure 5 illustrates several orientations of linear secondary opponent 
center/surround cells contributing to a resulting circular secondary opponent 
center/surround cell; 

[00024] Figure 6 illustrates a graded neutral density wedge covering the edge 
of a detector array; 

10 [00025] Figure 7 illustrates 3 examples of spatially coincident opposite-polarity 
center/surrounds, and their role in cancelling detector noise; and 

[00026] Figure 8 illustrates a single orientation of oscillation along a path 
sampling aligned slices of different textures and features, and their resulting one- 
dimensional center/surround crossing spectra. 

15 

DETAILED DESCRIPTION OF THE PRESENTLY PREFERRED EMBODIMENT 
[00027] The present invention is directed to a vision system, method and 
apparatus utilizing spatial oscillation. As illustrated in Figure 1, the system 
includes a primary detector array 12 having detector elements 14 sensitive to an 
20 input image signal 16, means 18 for inducing a spatial oscillation in the image 16 
relative to the primary detector array 12, and a filter 20 for filtering out those 
elements whose motions do not reflect the induced oscillation, therein removing 
detector noise events to provide enhanced image quality and simplified post- 
processing. 

25 [00028] In an embodiment of the present invention, the induced oscillation 
means 18 is provided by a swept-frequency sinusoid chirp 22. In an embodiment 
of the present invention, the system further includes a secondary array 24 of 
opponent center/surround detectors 26 grouped by spatial orientation to provide 
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multiple evenly-spaced orientations in the primary array 12 for receiving either 
positive or negative signal inputs at the center/surround detectors center, and 
receiving opposing input from primary detectors adjacent to a primary central 
detector, each center/surround detector fed by one or more detectors from the 
5 primary detector array, therein extracting improved de-noised, calibrated, 
contrast information and improved real-time spatial and velocity information. 

[00029] As illustrated in Figure 2, the method includes the steps of spatially 
oscillating an image in one or more dimensions relative to a detector array 100, 
and filtering for only those edges whose motions reflect the induced oscillation 
10 movement, to remove detector noise events to provide enhanced image quality 
and simplified downstream processing 102. 

[00030] In an embodiment of the present invention, the spatial oscillation step 
is provided by a swept-frequency sinusoid chirp 104. In an embodiment of the 
present invention, the method further includes the step of receiving either positive 

15 or negative interpreted signal inputs at the center/surround detector's center, and 
receiving opposing input from primary detectors adjacent to a primary central 
detector, each center/surround detector fed by one or more detectors from the 
primary detector array, therein extracting improved de-noised, calibrated, 
contrast information and real-time, increased accuracy spatial and velocity 

20 information 106. 

[00031] Spatially coincident opponent center/surround structures fed by the 
primary detector array are employed to remove spatio-temporally random 
detector-noise events. Spatially oscillating the image with respect to the detector, 
filtering only for those edges whose motions reflect purely the induced oscillation, 
25 and obtaining accurate phase information at edge-crossings accurately locates 
static edges. Spatio-temporal activity not suitably matching the space-time 
characteristics of the oscillation, or a Doppler shifted version of it, can be treated 
as noise. The texture isolation (discussed below), edge-motion detection, and 
noise removal processes rely on principles deduced from mammalian vision. 
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[00032] In order to visualize how the size of the detector can be made 
irrelevant in detecting objects smaller than the detector, one can imagine a 
circular optical detector divided into four detecting quadrants and illuminated with 
a spot of light falling entirely on the detector area. If the four quadrants are 
5 compared in pairs by determining ratios of energies falling on both quadrants, 
then the position of the centroid of the illuminating spot can be known accurately 
limited only by the accuracy of the comparator and not by the sizes of the four 
quadrants, which could be arbitrarily large. Given the self-calibration capability 
(discussed at page 11, herein) of adjacent pixels of a detector used in this 

10 manner, the timing of edge crossings can provide substantially improved spatial 
and exposure resolution when compared with the resolution that pixel dimension 
and bit-depth alone can provide. Spacings between detectors can give rise to 
hysteretic response depending upon the direction of crossing (analogous to 
backlash in a mechanical system), but neural logic can bisect such hysteresis 

15 giving true centroid crossing timings. 

[00033] To address spatial isolation of textures, edges and points, for a 
particular orientation of oscillation of an image upon a detector, center/surround 
crossings by various contrast-amplitudes representing a single slice of texture in 
that direction will yield a specific frequency spectrum of edge-crossing bursts, 

20 dependent upon the spacings and contrast-amplitudes within the texture slice 
and the frequency, or frequencies in the case of a chirp, of the generated 
oscillation. It should be noted that since the contrast contours of the texture can 
be determined by oscillation to a higher spatial accuracy at the detector array 
than individual primary detector size alone would indicate, substantially more 

25 accurate high-frequency information is available for texture discrimination than 
would be present from a standard static bitmap of that same image. 

[00034] In an embodiment of the present invention, a chirp in one or more 
dimensions can be put to particular use as the induced oscillation, since any 
object in motion across the field of view of the imaging array during the chirp will 
30 yield a Doppler shift with higher SNR, giving potentially more accurate velocity 



information. For example, a camera employing this method will only update "just- 
noticeable-difference" (JND) or greater moving edges, and do so instantaneously 
as they move, since motion of edges across opponent center/surround detectors 
provides an immediate response signal, and static objects can be optionally 
5 ignored after an oscillation has stopped, since they normally fade due to the 
memory time constant of the primary detectors. 

[00035] Utilizing Doppler techniques, accurate object velocities can be 
collected during oscillation by detecting the frequency shift with respect to the 
induced oscillation of objects crossing multiple opponent center/surround 
10 detectors, and noting the orientation of the sequence of crossings. For a 
particular orientation of oscillation, center/surround crossings by a slice of texture 
in that orientation will yield a specific frequency spectrum of edge-crossing 
bursts. Given more than one orientation and possibly different scales of such an 
oscillation, and grouping regions by their resulting oscillation spectra, texture 
15 regions can be efficiently isolated and identified. Textures, edges, and isolated 
points will have substantially different spectra useful for identification. 

[00036] The detectors can be incorporated with any reasonable "memory" time 
constant, or be programmed based upon local illumination upon each zone of the 
image on the array. Such a system will have access to free and accurate real- 
time velocity data, eliminating the need to calculate inaccurate motion vector 
fields from sequential, time-blurred and possibly interlaced frames. Velocity 
information determined from local motion on the array can be predicted and fed- 
forward in a velocity memory layer co-located with a similarly fed-forward 
contrast data in the contrast memory, following the object, permitting detector 
integration times to be de-coupled from faint but real contrasts of possibly fast- 
moving objects. Currently, motion vector fields are computationally expensive 
and require a latency of at least one image frame delay plus computation time, 
rendering them unusable in high-speed real-time motion applications. Information 
about velocities determined from the differences between frames seen in existing 
systems can be seriously outdated by the time it is available, even if such 
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information were accurate for the period being calculated. Also intra-frame blur 
makes instantaneous location uncertain, and noise is irreversibly integrated with 
signal, which also adversely affects velocity determination. Post-processing can 
use intelligent methods to clean an image, but once noise and signal have mixed 
5 this is still a form of guessing. 

[00037] In an embodiment of the present invention, using more than one 
direction of oscillation, regions with similar texture slice spectra in more than one 
direction can be grouped together; yet another plausible purpose of micro- 
saccades. 

10 [00038] The terms "primary" and "secondary" are hereinafter used to 
differentiate between the light-detecting first layer of "on"-detectors, and the 
second layer of opponent process detectors fed by the first layer. 

[00039] The invention's primary detector array will typically be monochrome for 
most applications, but can easily be extended to color or to non-light applications 

15 such as ultrasound. Each of the linear (symmetric) opponent center/surrounds of 
the secondary array of detectors are fed by at least three primary luminance 
detectors, which upon exposure to illumination of the primary detectors feeding it, 
will output a fast-rise-time pulse followed by a decaying analog level. This analog 
level will be proportional to either the luminance difference or the log of the 

20 luminance difference depending upon the application, between the center of the 
opponent center/surround detector and its surround. 

[00040] This level will have a characteristic polarity depending upon whether 
the opponent detector was on-center or off-center, while the surround will be 
opponent to either variant of center. Further, the level will have a time-constant, 
25 decaying after initial exposure, and will also possess a turn-on threshold that 
could be interpreted as a Just Noticeable Difference (JND), between the center 
and its surround. This threshold can be fixed or tuned in real-time to imitate 
human sensitivity, or any desired application-specific sensitivity. The JND can in 
fact refer to any achievable or required difference sensitivity for a particular task, 
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detection system, or image medium. Multiple opponent center/surround detectors 
of the same polarity chained in a specific orientation, and exposed to a moving 
luminance edge will output a train of pulses directly related to the apparent 
velocity of the edge, and superimposed on the contrast level signal 
5 representative of the exact times of successive center/surround crossings. 

[00041] During static periods, such as between induced oscillations, objects 
undergoing real motion detectable in the plane of the array can be selectively 
given attention by temporarily suppressing those detectors having detected 
stationary edges during the previous oscillation, greatly reducing the 
10 computational load on robotic vision systems. 

[00042] In an embodiment of the present invention, for the luminance case 
extendable to color channels and any other form of signal, the array is oscillated 
under the source such that each pixel in the array gets to see each source point. 
Given a fixed offset error at each primary detector, and (given the speed of the 

15 process) a fixed set of illumination values being sampled by the detectors, the 
resulting set of sampled values constitute a set of N equations in N unknowns, 
where the unknowns are the offsets. Given a small spatial sample for each 
detector such that the local luminance variation is probably small, the offset could 
even be a luminance-varying gamma curve, since over a small luminance 

20 variation the offset would still be a fixed value. Thus, the systematic relative 
differences between adjacent detectors could be determined, and the pixels 
calibrated against their neighbors by adjustable weightings to the opponent layer 
to provide real-time, ambient illumination detector self-calibration. 

[00043] As illustrated in Figures 3 and 4, an image is provided by a two- 
25 dimensional image detector array having an image focused upon it. The image is 
analyzed as a differential image in an opponent center/surround structure having 
multiple imaging pixels connected by logic circuits similar to structures and 
processes found in the human retina. Any luminance edge within the image 
crossing such an opponent center/surround detector will trigger a motion 



detection event or impulse at the instant that the centroid of an edge passes over 
the center of such a structure, along with a decaying level representing the 
instantaneous contrast between the center and its surround. 

[00044] In an embodiment of the present invention as illustrated in Figure 5 f 
5 the detector's circularly symmetric center/surrounds are composed of various 
orientations of rows of pixels connected as linear center/surrounds, contributing 
separately to the concentric circular opponent structure; similar to the ocular 
dominance column mechanisms in human vision. This will yield orientations of 
edges and directions of motions (assuming overlapping center/surrounds yielding 
10 quadrature signals for a motion). The attainable timing accuracy of the centroid 
of a spot crossing over an opponent center/surround detector will primarily 
determine the achievable "resolution" of the edge, and not the array's pixel size 
or the center or surround sizes of an opponent center/surround structure. 

[00045] If every edge that crosses such a detector is subsequently displayed 
15 with the exact relative timing of its crossing, then the perceived spatial resolution 
will be much higher than that viewed on a raster scan display of the same 
physical resolution. Alternatively, the implied finer path knowledge of the moving 
edge can be displayed as a temporal sequence of finer pixel crossing events on 
a higher-resolution display. This additional temporal knowledge at edges will 
20 yield higher spatial resolution information than would be indicated purely by the 
detector's pixel size, enabling the use of larger pixels at the detector to integrate 
more signal, and thereby yielding a higher signal-to-noise ratio (SNR) at a given 
exposure. 

[00046] In an embodiment of the present invention, since the image is viewed 
25 as a differential luminance map at the secondary opponent layer of the detector 
system, if the imaging array border is covered by reference-black then all 
luminance, and similarly colors, can be coded as spatially accumulated 
differences from the surrounding black reference across several center/surround 
detectors, or several scales of center/surround detector. This becomes crucial in 
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the human eye where there is no clear evidence of a low-pass version of the 
image for calibrating color and luminance, as is available in a wavelet pyramid. 

[00047] In an embodiment of the present invention as illustrated in Figure 6, a 
known neutral density wedge fading to reference-black is used to aid in allowing 
5 a gradual brightening near the edge of the field of view of the detector toward the 
center even in a uniformly bright field, permitting adjacent opponent 
center/surrounds near the edge to report differences kept within the range of 
each of the individual center/surrounds. In a multi-resolution structure such as 
the wavelet pyramid, this permits a higher dynamic range than any individual 

10 opponent center/surround cell is capable of providing due to the successive 
approximation nature of the increasingly finer scales. Equivalently, the local JNp 
threshold at each center/surround can be spatially and temporally varied to 
prevent saturation or under-illumination (recording the threshold to compensate 
at the output), or to reflect the characteristics of human vision, or to reduce 

15 information content in low light, high contrast, or high texture for uses such as 
reducing the computational load on any subsequent intelligent vision system. 

[00048] Ignoring texture, most slowly varying luminances in naturally 
illuminated scenes can be described by JND contours, which are only one bit 
deep by definition, similar to isobars on weather maps. Therefore, the dynamic 

20 range of each individual pixel does not have to be great in a multi-resolution 
opponent center/surround structure. This is particularly the case if each pixel's 
sensitivity is logarithmic. These may be the underlying structures in biological 
vision where the total dynamic range is huge, while the sensitivity to tiny adjacent 
differences is simultaneously high; yet individual neural detector "bit-depths" 

25 must be too small to accurately cover the entire dynamic range of the visual 
system. 

[00049] If dual-polarity pairs of opponent center/surrounds are used to cover 
the same spatial location, again reflecting similar human retinal structures, an 
edge traversing an on-center/off-surround will cause a positive impulse, while the 
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same event simultaneously detected in an off-center/on-surround will be 
indicated by a negative impulse. As illustrated in Figure 7, if on-centers poll a 
small population of pixels in a small spatial cluster, and off-centers poll a different 
small population of pixels spatially centered in the same cluster, with surrounds 
sampled analogously from an annulus of pixels around the central cluster, a 
single noise event at a given instant in any pixel contributing to a center or its 
surround will unlikely correlate to any simultaneous single pixel noise event 
contributing to a center or surround of the opposite polarity at the same location. 
Such a dual-detection process will cancel out most detector noise events due to 
the lack of a simultaneous response from the spatially coincident, opposite-sign 
opponent center/surround. It will also confer immunity to direct current (DC) level 
changes analogous to common mode rejection in a differential amplifier. 

[00050] In an embodiment of the present invention, the oscillation path can be 
precisely determined by employing a reference illumination spot aimed at a 
15 sample point within the detector, and subsequently feeding back the exact 
measured nature of the oscillation. Detector noise and fast time-varying image 
noise will have an extremely small probability of passing through such a filter. 
Also, luminance variations, which are seen to be a systematic function of pixel 
location on the detector (known as "fixed pattern noise", although it may be 
20 luminance-dependent) via use of the oscillation, can be nulled thus calibrating all 
the primary detectors relative to their neighbors. This is suspected of being 
another purpose for micro-saccadic motions seen in the human eye. 

[00051] Between oscillations, any real motion of an edge along a specific 
orientation of several center/surround detectors will cause a series of centroid 
25 crossover events that can be directly interpreted from the detector as a velocity in 
much the same way that a child dragging a stick along a picket fence will indicate 
speed information by the frequency of the stick rattling along the fence. Since the 
detector reports every center/surround crossing event, there is no missing motion 
data and therefore velocity and motion vector fields can be reported precisely. 
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[00052] In an embodiment of the present invention, if the opponent detectors 
indicating the latest positions of static edges found during the oscillation phase 
are suppressed between oscillations (by the detector memory time constant) the 
system can be made very sensitive to anything possessing real motion, since 
5 only these detectors will report motion during the periods between oscillations. In 
any intelligent vision application the isolation of known moving targets and the 
identification of their velocities are critical time-consuming tasks, which this 
detector structure can perform efficiently and accurately with little processor 
burden. 

10 [00053] Groups of these detectors in the array can be further chained in a 
multi-resolution structure with different orientations separately chained. 

[00054] As stated previously, the oscillation can be used in different 
orientations of the image to extract multiple directions of one-dimensional, high- 
resolution texture spectra over different lengths of opponent center/surround 

15 detectors at a given coarser scale for rough identification of similar texture areas. 
A coarse scan can group textures using a crude spectral comparison for 
purposes of locating approximate borders between different textures. Then, 
concentrating along these approximate borders, a higher-resolution texture- 
difference scan can be made using more sensitive criteria. This process avoids 

20 massive iterations or computationally prohibitive filtering, while permitting 
spatially accurate texture edges to be mapped. These texture spectra are derived 
from "pulse trains" generated by the chained opponent center/surround detectors 
having the textures oscillated over them, and will reflect their spatial resolution. 
By using a multi-resolution center/surround structure starting with a coarser 

25 scale, regions can be grouped by similarity, then at a finer scale near texture- 
difference borders previously derived from the coarse scale examination, exact 
texture borders can be determined using less on-chip processing. Thus, similar 
texture regions can be efficiently isolated and, given a suitable look-up table or 
other known methods, textures identified. 

15 



[00055] Scaling is logical since an arbitrary sample set of primary detectors can 
be used for a particular center and/or surround, and several different scales can, 
in principle, use any one detector variously as center or surround and as on-type 
or off-type. Scale can be employed to gather more light as above, and to verify 
5 an event in a region by its appearance at different scales in the same locale. In 
this case the selection of primary detectors being sampled is important, since it 
may produce an unwanted self-fulfilling prophecy by measuring an event at 
multiple scales using the same, possibly noisy pixel as evidence at various 
scales. Assuming judicious pixel selection for each scale at a given location, 
10 evidence of motion at a coarse scale with sufficient illumination can be used to 
adjust integration times and gains of pixels reporting to a finer scale, in order to 
facilitate extracting more detailed motion information. 

[00056] These logical scales can apply to various lengths and orientations of 
lines acting as opponent surrounds. This can be used to predict direction and 

15 velocity of objects crossing the field of view at various speeds. So, if one wants 
more velocity accuracy, then the firing rate of chained short lines being crossed 
is better. If sensitivity to a low contrast edge moving is important, then summing 
perpendicular to direction of motion will be useful. Thus, steered shape 
opponent-surrounds can be created to extract faint edges, and by spatially 

20 steering these shapes one can maximize the sensitivity to the direction and 
velocity of a specific stimulus. 

[00057] In human vision, it is believed that the spatial processing of texture 
occurs first at the retina, possibly to limit processing performed at the visual 
cortex, but that there is insufficient neural structure or bandwidth to handle the 

25 texture directly in the retinal image-space. Micro-saccades would serve this 
purpose by transforming the two-dimensional image structure into a few one- 
dimensional structures, such as neural pulse trains "mixed-down" to the lower 
frequencies that are handled by neurons. Texture can be differentiated from 
edges and isolated points given a particular size of center/surround region. As 

30 illustrated in Figure 8, the oscillation of a texture in various directions will yield 
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orientation-dependent noise spectra, while oscillating across a simple edge will 
provide a much narrower shaped band response dependent upon the orientation 
of oscillation, with an isolated point giving back a narrow spike for any orientation 
of oscillation. 

5 [00058] In an embodiment of the present invention, a stereoscopic pair of these 
detector arrays can be utilized to provide information such as safe landing 
locations for pilots based upon texture and flatness analysis, as well as 
instantaneous trajectories for path prediction from accurate stereoscopic 
velocities. A stereoscopic pair can also prove useful for the navigation of systems 

10 such as robotic arms, vehicles, or aircraft in three dimensional space for 
interception/avoidance and capture tasks since the array will report every 
detector element crossing event in real-time with accurate positions and 
velocities. This is a capability not available in prior art raster-based vision 
systems (which cannot report pixel-to-pixel motions, and suffer from motion blur), 

15 such systems processing raster-scanned images and extracting inaccurate and 
delayed motion vector fields, since a delay of one video frame alone in any prior 
art real-time system makes navigation and interception of moving objects very 
difficult for anything but constant velocity or low velocity applications. 

[00059] By intermittently adding a one-dimensional or a two-dimensional pre- 
20 determined spatial oscillation to the image or the detector array, real edge 
crossing events due to static edges in the image crossing these detectors (solely 
due to the relative tJscillation) will possess the spatio-temporal motion signature 
of the induced oscillation. These events can be filtered for, since the oscillation 
path is known or can be accurately tracked. 

25 [00060] This "threshold extension" technique used to reject detector noise, 
thereby increasing SNR, can be applied to radiology to permit a reduction in the 
dose of radiation required to produce an x-ray, and/or to bring diagnosable 
features up and out of the signal noise. 



[00061] In an embodiment of the present invention, a spatially varying 
detection threshold based upon localized image content can be employed to 
suppress texture, control entropy, or match human vision characteristics. The 
detector does not employ raster scanning, but instead reports the exact timings 
5 and orientations of edge crossings from one opponent center/surround to the 
next. Therefore, its spatial and velocity-reporting accuracy are functions of its 
opponent center/surround temporal sensitivity, enabling the use of cheaper and 
larger pixels with fewer pixels per detector array. Since actual velocities are 
reported in real-time from the array's support logic, there is no need to calculate 
10 processor-intensive inaccurate motion-vector fields. In an embodiment of the 
present invention, the systematic cancellation of detector-difference during the 
oscillation of the array provides for the self-calibration of all adjacent detectors. 

[00062] The invention utilizes the differential image provided by the 
center/surround structures to accurately determine position from the exact time of 

15 the crossing or phase of a contrast edge over such a detector. This can give high 
positional accuracy, fairly independent of the shape or size of the detectors/pixels 
themselves, enabling the use of large-area pixels with good light gathering 
capability with simultaneously high spatial resolution extracted from the time 
domain. The invention extracts a high-resolution image with a minimum of 

20 residual detector noise whether the image is optical, acoustic, composed of radio 
waves, x-rays, or any other medium, along with texture, motion and velocity 
information, all directly from a detector array and its supporting logic. 

[00063] The invention obtains a high-resolution, de-noised image directly from 
a lower-resolution detector by removing spatio-temporal detector noise and pixel- 

25 to-pixel differences due to differences in detector responses, while analyzing 
sub-pixel resolution motions. As well, the invention greatly reduces downstream 
computational load for intelligent vision applications by isolating moving objects 
while removing noise, and since this is applied at the detector level, in the 
absence of any active transmission or illumination it can be purely passive, and 

30 therefore stealthy. 
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[00064] Although the present invention has been described in considerable 
detail with reference to certain preferred embodiments thereof, other versions are 
possible. Therefore, the scope of the appended claims should not be limited to 
the description of the preferred embodiments contained herein. 
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